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"It will always beposslble to debate as to what outcomes Golleges should be 
trying to generate in their general education programs, what priorities th% 
different objectives should have, whicH of those such as here addressad oanr\ 
be 'taught deliberately/ and llk#qu#stlons. In this instance a siiabje grbup of 
col leges, which are known to represent Interests shared by a ta^^er ccrmpany, 
had the Ofaportunity to try to clarify what 'they most ^valued among the 
possible outcomes io be sought and assessed. Given the current challenge as 
to whether cairege 'educatlon \% really worthwhile and the extensive pres- 
sures to answer that question primaEJIy In terms of Job- and career-related ^ 
concerns, the choice made by thesescolleges and applied In this project Is a 
significant effort to clarify and revitalize the commitment to CQmplementary 
values. It reaffirms the idea that college education should be jCidged by the 
quality of life to which it contributes and that this qualUy Is a matter of the 
development of interests and concerns and of behavior ^applying those 
concerns in partipipation in the adult community." , 
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PuipQStV Thl American^ College Teiting Program (ACTl) organized ';the Colleg^ 

_ d utcom fe Me asures Pr olact ( QOMP) In- 1976, as a major codperative ^ffW^_ 
—rrr^y jj^i^^lyl^g g aria pgifTiBi^ 

ciariupport fo^ COMP has come from the Fund for the Imprdvem'ent orf Pdst- ' 
secondary EduGitiom(FIPSE); The purpoie of COK/fP is 10 design, djBvelop,^ 
validate, and implement asseisment instruments a 
: ' ; y arid evalMate the knowledge and skills (1) that undergraduate stud©nt%.are ^ 

; \ ; expected to acquire as aVresult of general or liberal education programs, and 
, . . ' (2) thatare Important tq e(^feG(rve funGtionlng iri ad .J 

'Assumptions The first major step In the project wal to identify, by sorting and analyzing^the^ 
large number, of goals college^ Jlat for their programs a set of cornpetenteieiv 
or outcome domains commoffiy held by^the partlcmating institutlorii md /v^ 
- ^ agencies. In attempting to define the major areas of general education kno^^^^^ 
edge and skills for which no adequate tes~tSfxlsted, COMP made a nurrib 
fundamental assumptions: ^ v . - / * 

' ' ■ - • COMP s.hould assess genferic college outcomes raffier than discipliVie- or 
content-based outcomes, \ J 

■ : • COMP sfeiould assess the college outbdmes which postsecondary igistltu- 

^iOna usually pxpect to result frc^c^!^he, general edu 
■\ their curriculums, ■ (-.^. -^ ^ ■ , 



COMP should assess the college, ptitM for effecti^^Tunc- 

tioning in adult roles^ ! C c \ ^/ ^ '' 

COMP should allow students the opportymty to demonstrate the full range 
of general knowledge and skills the^''pofsess. .( 



• COMP should provide an assessment package which possesses fac6 
vaiidlly for students, faculty, and external publics. 



Outcomes After extensive definition arid^r^iew by the twelve diverse institutions and^^ \ 
agencies participating in COMP, 'six areas were identified: communicating*^ ( 
^ ^ solving proWems, clarifying yalues, functioning within social institutions, 

using science and technology, and using the arts. Participants in the proJect\ ^ , 
believe that thli^list contains some, but not all, of ihe competencies or 
outcomes that should result from general or liberal education and that sho 
'■^ be importanmo effective adult functioning, Much^of the research of the ^ 

- project centers around an empirical exa/nlnatlon of these. beliefs. The mijor - 
general' knowledge and skill areas upon whicri the project is focused are: 

^ * , ' " - ^ / / / - f ^ * V = 

■ ■ ■' ' • ' " 

CommumlAting: Can send and receive Iriformatiorl in a variety of. mbd^ « - 
.(written, graphic, oral, numeric, and symbolic), within variety of settings^ 
] / ' (one-to-one, in small and large groups), an^ for a^ariety of purposes {fo\ ^ 
example, to inform, to understand, to persuade, and to analyze), = V > 
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V- ^ ' /* ^ ^olyfng ProfcleiTisi Can analyze a var 

, - \ . . ■ . ^ \ v'locial^p^sonal);^select or cfeat#iolutions to.^ 

k splarlfying Values^ dan identify^ne's personal vtilues'ahd the periona4 values 
^ of othSf^jnilvlduals; Understand how personal 'vdlues develop; and arftlyie 
the Ir^Jications of decisions made onftffe basis! of persoqally held values. 

Pun^tionlngh wlthli^ Soeial Institutions: Can identify Jhose activities and 
institutions which constitute the sqcial aspects of a culture (for example, 
governnrieniar a/id economic, systems, religion, maritaUand famiBal institu- 
tions, employment, and civic, volunt|ei\ and recreational organ izatiqns); 
. understand the Impact that social Instltutians have on lndi^«dual#in a culture;. 
. r I . an^.pnalyze one's own and others' personal functioning withi^ social Ins 

' ^ ^^^^^ ^ . ' H ^ ' . 

Uslii^ SGltncQ and Ttchnology: Can identify those activities and products 
which consmutj the scilntlfic/technold^ital aspects of a culture (fpr^ 
exarni^le, tran^fcrtation, bousing, *%ner(gy, processaa fooa, clothing, health 
mairttenafTCff^ntertainment and recreatiorT, mood-altering, nationar<Jefense, 
€|^ommunlcfltionp and data processing); understand the impact of such 
^ activities and products pn the individuals and fhe* physical environment |n a 
culturerand analyze the uses of technological iSTooLjcts In a rfui«ire andpne's 
/ . ' ' persorial use ^f such products, ^ * 

■ : ' 1 Oiing tht'Afts: Can identify those activities and prod 

i artistic a^^ctsof a culture (fprexample. gpphic art, music, drann^^ 

^dance, sculpture, film, architecture); understand the impact th4twt, |Vi its 
» ' various forms,, has on Individuals in f culture; analyze uses of wofrk^ of art 

^ " ■ ■ wifhio a tf^ture and one's personal use of arti, ^ 

. - ; y ■ - , / ^ , ^ ^ A - ^ . 

^ , ■ - - / . \ / . ^ ^ K ^ ^ V / , ' 

. , Assisimtnt btilgn In its fi^Ltwo yeaf|jgf researcfj and dd'^eloppentV^^MP has Initi^ei^three 

I ' . diffeKenrapproacheS/tb the as^essme ' 
^- * \ areas that were defiried. Thff key featurd^ of this asseppjia 

V ' . ^ determined In accordatee vyjth assumptlQns%b 

and the nature ©f the^^owledfle and skills to\be'a 
. . ^ . ' ' ^ -are: ^ ' ^ ^ . ^' ^ ^ ^ " ' 

-^k /r^ , ^ " ^ ■ . ^ ^ ^ . ^ / 

/ s _y ' - • thd employjTient of realistic stiffijUlus material drawn, from the adult puW^ 

^ ' ,^ / * donriain, such as 'television documentaries, articles, adverti 

■ " ' ■ stories, charts and diagrams fro^ magazines an0 nev^sp^^^ 

■ ^ , ' . . editors; business memoranda; di^U8sions;''telephone calisi! speeches; ifrt 

^ ' , ^ ^ ' posters and prints; musical recordings; films; / ^ s-i^^ir 

^ = ■ • the use of an open-response format (in addition to the^usuahmultipfe- 

choice questions) which requires students to provide their own answers 
' rather 'than simply selecting one of several answerr supplied' 

\ 1 / • • development of ipjIatioRs that Require the application of k 

"v . ^ * ' skills to* problems aftd issue^ commonly confronted by adults; 
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• techniques for the evaluation of student responses which allow faculty to 
■^ase t heir judgmen ti on standardized rating icales ; , / ' 



• a modLilar assessment^ package Jrom whlch^stltutlons may select those 
portior^s 6f*the materials which meet their needs; , ^ ^ ' > . 

• .multiple perspectives from whiGh4o assess each student response in order 
to yield Judgftients about at least two areas of knowledge and skill. For 
example, a student's writing sampr© might be ^ssesfed to fctermine both^ 
skiJI in sqlving probJiBms and knowledge aboul the use of science and^ 
technology, , t ' ^ * V 

Considerable effort ha& been' made to develop "^struments^that are content 
fair; process fair, and efficient. Yo be fair, an assessmenfof content should 
cover a broad scope of Drocesses, just as^a fair apesbment qf process should , 
i cover a broad scope ofjcontent In COM P, efficiency is gained by ijptlrrelat- * 
jfig the content" and process-relataci areas defined above. This interrelation- 

/ship is illuttrated in the following *sart. Tha three content-related outcome 
doma^ms assessed b^y COMP (Funetiohing within Social Institutions, Uplng % 
• Science and Technology, and Using the'Arts) are shown at the top of th| 
ch^art. The three prooess-related outcome domains (GoYnmunicating, Solving ^ 
Problems, apql Clarifying Values) are aWown at the left side of the chart. Each 
of^the nine areas in the chart represents, the interrelationship of content and 
process skiljs. This approach to definitlboand assessment of genial educa- 
tion outcomes ife further madte ^ost-effectiye by designing modularized^ 
instfymentS'So that specific |reas and compoffents can be seleoted t9 fit the 
.^iqiie ne^m of a spepific insi^tibn, * ^ . ^. . 



X. 



Cdmmuniaating 



Solving 
Problemi 



Clarifying 



Functioning ^thln 
Social Inititytlons 



V 



Cising Science 
and Taehnology 



Using 
the Arts 



Corrtmunicating 
'abpi^ Social 
Institutions ^ 

fl Solving Social 
Problems - 



Clarifying Social 
Values 



Contmumcatlng 
about Scien^ and 
Technoiogy 



Solviing Sclemific/ 
Technological 
Problems' 

Clarifying Scientific/ 
TecNielogicaj 

Values >S 



Gommunlc^lng 
about the ^rts 



-Solving Artistic 
Problenns 

Clarifying Artistic 
Values 



A detailed description of the assessmentidesign, including specification of^ 
"stimulus and response modes, is inclUdedln the COMP 1977 Annual Report. 



COMP/ACT The three iristruments developed to ^ite are: (1) a Measurement Batteryp 



AMessmtnt- Instruments 
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(g) 4r% Ob) ecti ve Test^ akcl (3V an Activ Ity Inventory. Questioni in th^Maa" 
suiwipint Battery and thf Obiective Test are based on ttleyislon dooumen- 
^ , tarles, repent magazine articles, ads, *shOTt stories/ art prints, music, 
discussions, and newscasla. In QSfie'^ of the Measurement Battery, 
^ ^ participants view these materials and then teS^ond with short written 
an jwSrs, y ith some longer esfays Written In the form of letters, and with some 
oral resp^onses which are a^dlotaped or videotaped, in the Objective Test, 
pailici^nts Respond to thf same materials through ^innovative fnultlpre- 
chol^ qurttions.^The Activity Inventory uses a/fnultiple choice format to 
assesi th# qfuality and quantity of parjScitsation In out-of-clMs activities 
related to the sixi^utcome areas. 

Petalled descriptions of thesi three irtstruments appear ' In the COMP 
Prospectus for 1978-79, 



Institutional Use of COMP Increasing numbers of postfecondary institutions have some to recognize 
' Mattrlais the Importance of carefully spegifyrng the ^general knowjedge, skills, and 

t attitudTes needeb by theingf^duates ^o funDtion effectively as adu 

accurately gssessi Ag ^udent achievement of f uch general knowledge, skills, 
^ . and attitudes. Institutions find*Jhey need such assesiment information in 
order tdK i 

• assure ^at students are reaching general or liberal education goals and 
are rgfieiying appropriate '^edlt and recognition for their achlevemerrt of 
jth^se goals; . ' 



\ 



• 1 ' V 



shape the curriculum so as to'incluj^^ those leaning activities which will 
help students obtairtthe knowledger skills, and attltudci crucial to adult 
functioning; " \ ■ ^ 

^ / ^ " 4 ' ^ 

hiTp* students plan the general or jy^ral education programs that best meet 
eir expectations and pribcjtle's, fis well as those of society; 

assure the consumeire and supporters- of postsecondary education that 
general and liberal eaucation 'programs effectively Impart at least some of 
<he general knowledg^and skills students neid to function effectively in 
^ \ fedult Society, ^ , r - /; 



Asiun^ptioni about Use The nontftditional nature and^desigh of 'COMP materials means that 
= A 1 specialized tr^'ning , is required for their i^se and Interpretation, COMP 

J \ _ 'provides insti|utibns* with a set of services to aid in defining and miasuring 

the overaU'^outcomp'of a^ general program of education. Specifically, these 

, . \ % servlGes include: . 

f ^ i • assisting ift outcome de^nition and examining the fit between eac 

S . tib^ general ^ucation outcomes and COMP assessment instruments; 

. \ 
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^ . ' *' • advising on appropriate and inappropriate, uses and jn^rprete 
^■^ COMP ass e ssm e n t m ateri al s; >^ 4 , 



• counseling In th^ettlng of reasonable assfte^sment standards; 

• assisting in the developmeni of ■supplemental assessment materials to 
fetter assess each institution's unique general edurttlon program. ' 

An institution exploring ule of CQMP j/vould be ex^jfected to:, ^ 

• consider the^valldity of COMR with regard to its own program objectives 
and instructional activities; ^ - * . 

• try out the basic COMP materials to estabNshk local norms and facilitate 
rational standard .setting; ^ 

weight scores or augment areas assessed to tailor C^OMPjo the unique 
ne^ds and purposes of the institution;. ^ 

• studv^e eftectiveness of programs intended to develop the skills to be 
measu^WDefore using COMP for certification, Crediting, or placement of 
individual students in these areas, ■ 



Slaps in Davelopment , The plan for development of the COMP Assessment Package involves.a mqjor 
and Validation commitment of time and effort by AcT staff over an 8-10-year peridd, A key 
*r ' ' feature of the COMP research atid development effort has/Deen the 
substantial* assistance provided by faculty and adminiltrators at the 
participating mstitutions and agencies In (1) the identification of outcomes/ 
(2) the development of COMP materials, and (3) a series of field research 
tryouts of the assessment instruments with students and nonenrolled*adults. 

The initial field trials in spring 1977 tested thte feaslbllrty of the assess^ 
approach described in this brief'overview. It yielded useful Information about 
r ^ the face validity of materials and pvidence th^t the outcomes Identified were 
^ indeed addressing skills developed in colieg# programs. The materials - 

, ' yielded results not refleeted in traditional measures, such as CPA, and not 
f V accounted for by maturation' or abiltty alon&. 



Validity A pilot validity study was conducted in summer f977 with mature adults 
Identified as functioning effectively in three areas relevant to portions of the 
assessment nrjaterlals. This exploratoVstudy provided support for the belief 
' that the domains assessed by COMP are important to effective fitnctioning In 
adult roles. Four more adult studies conducted in fall 1977 and spring 1978 
yielded similar rasults. A series of comprehensive adult studies is planne*to 
-^-^ documertt more fully th% relationship between g#^ral education outcomes 
and adult functioning in a variety of roles. The first of these expanded validity 
studiep was begun in August 1978, ' % 



er|c 



fteilablllty Field trials of revised and nawly developed assessrrienif materials weri ^ 
c onducted In talL1 977 an d spring 19Z8. These studies almed^to determine the. 



objectivity and reliability of various components of the mattrials, especially 
tha^egree to which faculty could agree on* the evaluation of student , 
responses, ACT staff were reassured and encouraged by the findings Jfrom 
these trials. In summer 1978 further stui^les w,ere conducted to eKplort the 
degree 'to which training of faculty in trte use of scales to rat© student 
'responses would yield qven higher levels of agreement. The purpose of the 
remainder of this report is to describe these validity and reliability studlsfe 
conducted in 1977-78. 

The COMP Prospectus for 1978-79 describes-further plans for refmement and 
validatibn of the COMP assessment package and invites participation by 
postsecondary Institutions in experimental use of the materials. The final 
section of this report also contains a formal review of the first two years of 
COMP efforts by Morris T. Keeton, ch^airman qf the COMP.Advlsoi7 and 
Evaluation Panel, ^ 



I' 
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RESULTS qNImAJOR 
STUDIES GOilDUCTED 
IN 1977-78 



Fall 1977-Fleld Trials 
of the COMP^KCT 
Mea$uram@nt Battai^, 
. Fdrrti II 



TheCOMP/ACT Mpasursment Battery; FoPm IK isan mstrument deiigned^o 

-measure^ ^he-fbllltyfcf stydi 
Education skflls and knbviMedge thought rmportant for effective functioning in 
adult roles. Ct Includes ^0 questions cluSte/ed anound 15 actiyme^ that usp 

* realistic stimulA materials drawn ^from the adult pubMc dqmalh {tbleylsion 
documentaries, ads, magazine articles, music, etc). The questions^equire 
students to provide th^ir own answers^ fncluding nine .minutei p* laped 
responses, rather than rrTerely' sel|cting from among given, answers. The 
degree^of adequacy of each response Is t^en evaluated by four trained faculty 
evaluators. expert in the various domains ass^essed, using standardlzfd rating. 

- scales. These 60 scales alLow faculty to discriminate fl^e levels of proficiency 
in each^tudent response. Administration and scoring afptloris are availably so 
that a college can f if the Battery to its own unique general educafion objec- 
tives. If the entire Battery lo used, the maximum total score pdssible is 240 
points. ' ^ > ' * 
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In addition to a total score, seventeen subscores ar^ derived for the COMP/ 

AOT Measurement Battery.! Table 1 shoW^^how these subscores relafe'to 

the' basic design of the instrument. The first six subscores, each iiased on\. 

18 to 20 questions in the Battery, relate to the six outcome domains of general*, 

education knowledge^and skills, outlined and defined on pft. t-2, Besides the 

domain score for Communicating, two specific Communicating subscores. 

Speaking and Writing, are produced. These are defined in Table 1 and are 

based on nine minutes of taped responses and on sixty minutes of written 

responses to produce three letters. The remaining nine subscores represent ^ 

thi application or the skills of Communicating, Solving Problems.^ arid 

Clarifying Values in social, scientific, apd artistic contexts. These nine scores, 

each based on 6 to 8 questions in the Battery, are defined Jn Table 1 , Thus thi 

. use of the entire Battery Vields a comprehensive assessment of up to two 

vears of an undergraduate liberal arts program. These 18 scores reflect, in 

|um, the ability to apply general education skills and knowlddge in a wide 

variety of contexts. ^ ' ^ - , 

' ** ^ ■ ^ ■ * 

Administration time for the entire Battery requires about six hours, from 
students. All ^ut nine minutes of the Battery can be administered to groups of 
students. The remaining nine, minutes/ which involve vld^taping of oral 
responses, mifet be adminisjered to one student at a time. If audiotaping is 
used Instead of videotaping, language lab facilities can be used to tape 20-30^ 
students at a time. It theh takesabout one hour per student for four faculty 
members to evaluate the student's written and oral responses. For example, if 
100 students were tested, it would take a total of 1 00 person-hqurs for the four 
faculty members to complete ival'iiatron of the 100 students. If only parts of 
the Battery. were used, less time would be required of sftudents and Jaculty. 
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TABLE 1 , ' 

Subsoorea Derived from the COMP/ACT Measurement Battery 

_ ^ , » ' 



i; FUNCTlQNrNGr 
WITHIN SOCIAL 
INSTITUTIONS^ 



2. USING SCIiNCi 
AND TECHNOLOGY 



3, USING THi ARTS 



C COMMUNI- 
CATING ' 



7. Speakings ability to establish a human relationship With an 
audience, and coftcern about delivary (animation fluency, 'poise) ' 
as well as discourse (organization of ideas, examples, detail). This 
score fs based "on _nine minuies of taped responses to questions 

^ "dealing with social institutions, ^cience^and leohnology, and art, 
for which advanced preparation is allowed. , ^ * 

8, Wrlllnq: ability to send information to a particular audience with a 
focus on persuasiort in which points are developed in a clear and 
controlled way and dSrrect and lively use of language is 
demonstrated. This score is. based on sixty minutes of writing 

to produce three letters= 



9. Communtcatlng 
about Social 
Institutions; 

ability to send and 
receive information 
(including numeric 
and graphic material) 
related to activities 
and institutions which 
constitute the social 
aspects of a culture. 



12. Cor^municating 
about Scitnca 
and T@Ghnology: 

ability to send and 
receive information 
(including^numeric 
and graphic material) 
reiated to the 
scientific/ technological 
aspects of' a culture. 



15, Communicating 
about the Aiis: 

ability to send and 
receive information 
(including numeric 
and graphic material) 
related to the artistic 
a'specte of a culture. " 



5. SOLVING 
' PROBLEMS 



10. SolvIng^ Social 
Problems: 

ability to define 
problems of 
functioning within 
social institutions and 
select approaches to 
solve problems, 
generate solutions, 
collect information, 
check logical 
consistency, select 
a good solutioci, and 
evaluate the-process 
by which a problem 
was solved 



13, Solving Scientific 
and Techngiogicai 
Problems: 

ability to define 
problems related to 
scientific products and 
the use of technology 
in a culture, select 
approaches to solve 
problems, generate 
solutions, collect 
informatian, check 
logical consistency, 
select a good solution, 
and evaluate t^^ 
process by which a 
probl^i was solved: 



16. Solving Artlitic 
Problems: 

ability to define 
problems related to 
art and its expression 
and use in a culture, 
select appKoaches to 
solve problems, 
generate solutions* 
collect information, 
check logical 
consistency, select a 
good solution, and 
evaluate the process 
by which a problem 
was solved. 



CLARIFYING 
VALUES 



11. Clarifying Social 
Values: 

ability to identify 
one s own social 
vniues and those 
maior values and 
issues faced by - 
others in 'daily 
adult life in 
one s own and 
other cultures, 
understand how 
values develop, 
and analyze the 
implications of 
decisions made on 
the hasis of those 
valijes 



14. Clarifying 
Scientific and 
Ttchnoiogic^ 
Valuti: 

ability to identify 
one's own scientific/ 
technological values 
and those major values 
and issues faced by 
others in daily adult 
life m one's own and 
other cultures, 
understand how 
values develop, and 
analyse the ' 
ifnplications of 
decisions made on the 
, basis of those values, 



17. .Clarifying Artistic 
Values: 

ability to identify 
one's own artistic 
values and those 
major values and 
issues faced by 
otfiers in daily 
adult life in 
one's own and 
other cultures, 
understand how 
values develop, 
and analyzo the 
implicating of 
decisions made on 
the basis of those 
values. 



fi 



Seven instltutionf participated in the fall 1977 fierd test of the COMP/ACT 
Measurement Battery: Brigham Young, University, Colgate University, Florida 
A & M University, Mars HiH College, Michigan State University^ Our Lady of. 
the Lake University, and the University of Nebraaka-Lincoln. Each of these 
institutions was asked to identify 45 freshmen and 45 seniors with equal 
representation from the social sciences, natural sciences, and arts/ 
humanities. All student participants were volunteers. Not all the colleges were 
abile to" fill their student quotas, with two colleges providing about one-fourth 
and another two about .one-half the numbers requested, Faulty recording 
equipment and inclusion of sophomores and juniors further reduced the 
usable sample at two institutions. The intended and actual distribution of 
sttidentB is shown in Table 2. The^ombinea sample is representative of the 
. diversity In' .higher education and thus serves the purposes of the field test. 
Students involved at an individual institution, however, are rfot necessarily 
representative of all students at that institution. ^ 



TABLE 2 

Sampling Plan and Numbers Obtained iri Each Category 



FreshfnEjri 



Totals 



Totals 


Soaiai Sclenoes 


Natural Sciences 


Arts/Humanltles 


Intended 


Actua! 


Intended Actual 


Intended "Actual 


Intended 


Actual 


315 


140 ' 


105 56 


105 45 


105 


39 


(40%) 


(40%) 


(32%) 




(28%) 


315 


154 


1 05 64 


'105 42 


105 , 


48 


(52%) 


(42%) 


(27%) 




(31%) 




294 


> 120 


87 




87 




(40%) 


(30%) 




(30%) 



Background Information of the foHowing type was requested for all stu<dents: 
sex. aqe. ACT Composite score or equivalent, and total GPA (for seniors 
only). A summary of these datr- is shown in Table 3. 



TABLE 3 

Student Sample Background Information 







Sex 




Age 




ACT Composite* 




Men 


Women 


19 or below 20-22 23-30 31 + 


1-15 


16-20 21=25 26-36 Mean 


Freshmen 


42% 


58% 


94% 


5% 0 1% 


15% 


16% 37% 31% 22.5 


Seniors 


49% 


57% 


1% 


59% 23% 16% 


10% 


16% 31% 43% 23 J 


Totals 


4C5%' 


' 54% 




23% 12% 9% 


13% 


17% 35% 36% 23.0 



GPA (Seniors only) 

0-1 00 1 01=2,00 2.01-3.00^ 3.01-4 00 
0 4% 35% 61% 



*ACT seoros wqro availablo lor 131 froshmen and 96 seniors 
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Within the total group of 294 freshmen and seniors, various subgroups were 
weil represented, The sample was composed of 48% freshmen and 52% 
seniors with^no significaKt differences in ability by grade level or major. The 
mean ability of the sanr^ple, based on ACT Composite or SAT Total scores, 
was high, above two-thirds of entering college freshmen nationally. Slightly 
more students represented the social sciences (40%) than natural sciences 
(30%) and aVts/humanities (30%). Freshmen were predominantly 19 years of 
age or younger (94%). Ninety-nine percent of the seniors were age 20-22 
(59%) or older. It was determined that all but one of the institutional groups of 
participants contained comparable subgrpups equivalen^in ability across' 
grade level, major, sex. a/id age. The combined grou^ of freshm^en and 
seniors is judged as comparable and well-balanced. However, since each 
institution's sample was quite small and was probably not representative, 
caution is urged in interpreting results for Iridividu&l institutions. 



The results from the spring 1977 field test of Form I of the Battery, reported in 
the COMP 1977 Annual Report, were replicated and expanded by findings in 
the fall 1977 field trials of Form IL These results clearly reveal the dis- 
criminating power of the Battery and indicate that this type of asssssment 
holds a great deal of promise. The following sumrriarizes findings to date on 
the C0MP/ACT Measurement Battery: 

• The outcomes identified for the project appear to be related to the educa- 
tio^at^^prog^ams of the participating institutions and unrelated to simple 
maturation. The results of the first field test show a clear progression of 

^growth across four years of college, as reported in the COMP 1977 Annual 
^^port, p. 9. The results of the.fall field test for a new form of the BatteTy 
reveal highly significant differences in performance between freshmen and 
senior means. Seniors scored higher than freshmen on the total Battery 
score and on all 17 of the subscores derived for the instrument. In both 
field tests there is nearly zero correlation between age arid total COMP test 
^ score. No significant differences in total scores were noted by sex or major 
field. A study described later in this report comparer college seniors ^in 
programs that include general education .with vocational-technical 
students of comparable age and ability in programs that do not include 
general education. While the latter obtained substantially lower scores on 
the Measurement Battery than college senjors, a sizable minority could be 
judged as already possessing general education skills and knowledge at a 
collegiate level. 

• The Battery appears to be measuring abilities not measured by college 
grades. Seniors' total scores on the Battery have shown a significant, but 
small relationship (.20 intercorrelation with Form I and .35 with Form II) to 
their college GPAs. 

• The total Battery score differs from measures of general aptitude and 
ability. There is a clear relationship between performance on COMP and 
ACT Composite (or SAT Total) scores for freshmen (.67 intercorrelation 
with total score on the Battery), However, this level of relationship 



accounts for the variance IrrperfOTmanceof only 45% of the students. Most^ 
of the differences In performance cannot^ be attributed to differences in 
general ability, 

The Battery does seem to be gieasuring abilities relevant to functioning in 
adult roles. Adults who have been selected on the basis of significant 
participation in a particular comrnunity role (e.g., solving technological 
prgblems) show a strong tendency to score highest in the correspQRdjng 
area of the Battery than in other areas (statistically significant beypnd the 
.001 lever). Further discussion of this finding appears in the section of this 
report titled "Mature Adult Performance on c6mP/ACT Instruments," 

Data collected Indicate that the Battery may^be culture fair. The fairriess of 
a test depends on many factors, some of which are^yet to be examined. 
However, faculty from a variety of racial/ethnic backgrounds helped to 
choose stimulus materials, items, and rating sca+es. Local faculty, 
evaluating students a| institutions where minorities were represented, felt 
materials and scales were appropriate. The students themselves, at ^ 
institutioVis with high minority representation, responded lavorabiy abput 
the fairness of the Battery, . 

Some limited data is also available about the performance of various 
groups. Freshmen at three institutions (for whom ability scores were 
available) were compared; 

A predominantly white institution (N - all white) 

A predominantly black institution ^ 24: 23 black, 1 white) 

An institution with high minority representation (N ^ 24; 46% white, 50% 

Hispanic, 4% black) 

Mean ACT Composite scores for the three' groups were not significantly 
> drfferent (22.96, 19.96, 19,75, respectively) and the range of scores was 
similar. 

« 

Mean total scores on the Battery were essentially identical (87, 85, 86), 
While the mean ability of students from each of the three colleges was 
above the national average for their ethnic group, no one group appeared 
to be at an advantage or disadvantage' on Battery total scores or sub- 
scores, ^ - ^ 

• Rating five levels of proficiency for each of the 60 Battery scales allowed 
some degree of success for most students. If le^i 5 is considered failing, 
90% or more of the students provided a passing response to almost half 
(29) of the 60 questions. Over three-fourths of the questions were "passed" 
by more than 75% of the students. Of the nine questions for which fewer 
than 50% provided a passing response, only one could be answered by 
fewer than 28%, While some level of proficiency was demonstrated by most 
students on most Items, the Battery had a high celling which allowed for 
measurement of mature, experienced adults. The average senior scori on 
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the 18 scores of the Battery ranged from 42-53'%*orthe maximum possible, 
except for two areas: Clarifying Social Values,' 63%; and Solving Scientific 
Problems. 38%. 

Standardized rating scales enable faculty to make objective and'rellable 
Judgments about levels of performance. In the^irst field test (Form I of the 
Battery), where faculty rated responses from A-F without scales and using 
their own criteria, an agreement of 64% was obtained. With rating scales 
developed for the. second field test (Form II of the Battery), 74% agree- 
ment was achieved on the pass-fail standards selected by each Institution, 
Revised rating ^scales, developed in spring 1978 for readminlstration of 
Form 11 at two institutions, produced an agreement of 83%. These levels of 
agreement are quite adequate for an Instrument to be used in program 
evaluation. 

Further refinements in scales and scoring procedures will assure levels of 
agreement to allow use of Battery subscores in making decisions aboul 
individual students for purposes of placement, advising, awarding credit 
for experiential, learning, certifying for graduation, or screening for 
advanced training or employment. As institutions have shown a pref- 
erence for making Judgments of the level of proficiency rather than pas- 
sing at a selected standard of performance, other statistics for detarmin- 
ing reliability (such as the Pearson correlation) are relevant. Based on the 
revised rating scales for Form II, a total test score Pearson correlation of 
.90 was obtained= The following test-retest study reports on this informa- 
tion and also provides evidence of the stability of student responses.^ 

Faculty and student reactions lo the Battery have been quite positive. 

The Battery is considered cost-effective to administer an4-"score. At 
present, typicg^l costs per student average about $'25, including ACT staff 
time and travel for faculty training, faculty released time for scoring, 
administration staff time, materials preparation and rentals, and computer 
processing. Not Included In this figure are research and development costs^ 
or indirect costs to a participating Institution^ 



A Tesl-Retast Study A tbst-retest study of the 6t)MP/ACT Measurement Battery was conducted in 
of tha COMP/ACT March 1978 attwo institutions.with those student^who had participated in the 
j* _ _ : ^ ^1 Bgfi^ru * November 1977 field test. Designed tp permit evaluation of as many sources 
Maasuremeni ^iiiary, measurement error as possible, the study aimed to determine the reiia- 
. Form II ■ i^jji^y measurement materials in conditions approximating those of actual 
: .. use. ■ , ' 

One issue explored was the stability of student responses to open-ended 
. 'items. The interval of 14-16 weeks between test dates was sufficient to expect 
learning to have occurred which could affect the relative ranking of students!* 
f ' ^ Another idsue, also related to open-ended items, concerned the reliability 

' : with which rating scales were applied to judge the adequacy of student 

responses. For th0 first test, faculty evaluators at^he ACT National Office 
rated the responses, while for the retest, faculty evaluators at each of t^ie two 
institutions did the rating. Ordinarily, reliability pf taters is determined by. 
, ^: , ' . ^^comparing the digree 0/ agreement of different jddges rating t^ 

■ ■ . student responses. Here ratings by one set of judges of the first test 

responses are compared to ratings by a second set of judges of a second seV 
' . of student Vesponses— the same students' narrative responses to a second 

administration of the Battery.^ - " ' 

There were several other sources .of potential measurement error, including ' 
possible variations in test administration gonditions and instructions. Student 
motivation and effort to take this six-hour test twice may have affeqted perfof- 
manc^: Stu^dent volunteers took the first test, pn the retest, they were paid 
p $3.00 per hour ar\d encojjraged to try for higher scores. They were not, 

however, given resuits of their first performance until after the retest. 

' \ At one institution 29. of the original 41; students (71%) participated in the 
retest. At .the second. 41 of the original 46 students (89%) partiqipated ' 
Approximately th^same proportion, and about equal numbers, of freshmen 
qnd seniors at each institution took the retest, but about twice as many 
wqn^n as men were represented. On eaph test student responses to each 
question were rated on a five-point scale; these ratings vvere then summed to 

^ produce a total score for the Measurement Battery, For each institutipn, total 
scores for the November test were coVrelated with retest total scores to deter- 
mine whether the students ranked in the same order eagh time. At one institu- 
tion this reliability coefficient was .90f at the other institution it was .89. Such 
^ coefficients are unusually high for open-ended assessments of-this nature, 

where results over .70 are generally considered good. In view of the many 
sources of measurement error outlined above, they are remarkable, These 
results suggest that the 'Measurement Battery cSn be used to make judg- 
ments about individual students and that confidence can be placed in the 

' reliability of the total assessment procedure.^ " 

Correlation coefficients for the Battery's six mijor^area subscores were alsq 
- quite high, as Table 4 indicates: at one institution all correlations were ,78 or 
above; at the other institution, one correlation'was .66 while the remarning 
coefficients ranged from J3 to .88. These high correlation coefficients are 
adequate for using and interpreting results at indlviduaL and group levels. 

^ ^ ' ' ' : 13 



TABLE 4^ 

Teit-Retest Correlations oi Measurement Battery 
Scores for Two Institutions < 



Instituti^h g 
Institution 2 



fecore 

.90 
.89 



Functioning 

In Sociai 
Institutions 

J9 
.73 



Uiing 
Scitnoe 

J8 ^ 



Using 
the Arts 

,82 
.75 



Communi-^ 
eating \ 

.80 

= 79 



Solving 
probtemi 

^ .82 
• .88 



Clarifying 
Values 



i84 



Cef^iations for t^e, other subscores (except fdr Writing and Speaking) vvere 
generally, thd .60 Iqvel and are a(^equate for ajsin^^these subscdras^ to- 
evaluate^pTOgTarrrs. Subsequenfly, specja^ effort was made to^ refine the pro- 
cedures for evaluating writing and s^eakmg skills. As describe^ in tha last 
section of this report, the resultant reliability coefficients for judgments made 
Jn these two areas have, been increased to the .80 level. 

Another feature noted in the test-retest study, was variability injmean level of 
performance. At one institution mean scores renrained ejuite-, stable. At the 
other, mean scores increased by about 25%, with seniors showing' greater 
increases than freshmen. Further studies fViust determine ^^hether this 
increase in level of performance^i^s due to ^learning, to differencBs in test 
administration and rating, or to an educational eKperiehce within the test 
itself. ^ ^ 
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A Proxy Measure for A, study . of *the characteristic of the CwlP/AtT Objective Test was 

' • the MeasurernenV conducted in MarchH978^t five jnstitutons, using students v^ho partmfpat^d 

Batterv the '^i the November 1977 field test of the GuMP/ACT MQaSLj^ement Batt^/TJje 
'VrMmo/XnT riK' ■ ■ Obj©c|tive Test is the product pf aa effort id create proxy measuries for 'all* 

GOMP/AC T Objective components'of the^MeasuFement eaftlry. /Because this field test constitutes 

Test, Form | the te^'s first use. it offers only prelihiinary evidence of usefulness gte^^roxy 
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measure. 

The Objective Test contains 63 myltiple-chdce questions requiring appli- . 
cation of general knowledge ^nd skill to realistic adult situations. Each ques- ^ - 
tionXhas^two correbt Qptipns and two incorrect options. Selection of an 
incorrect option by a stu.dent results in a poiht'bein^ubtracted botKfrom the ^ 
aubscore to which the^u'pstion \s, related and from the total score. Tffe range 
0^ possible totaLscores on Jhfftest Is from -12610 +126 (i,e., 262 poyhts), and 
all scores are adjuste^d\*td * refled^t positive scored only. Both iterris and' J 
corresponding subs.c6res^are designed to paraHebwose of the Measurement 
Battery. Participant responses are recorded on Imachine-scorable answer 
sheets. Test administr^tior| requires^ two and pne-halfp hours. As with the - 
Measurement Battery ,/thefe are administration 'and scoring options to make 
the tesjr modular. '< 

'FlveJnstitutions particrpBted fn the firstf ield test of the Objective Test, with a 
total of 180>{76%) of those sti^ants wmo took the Measurement Battery iri 
Nbvember 1977 participating W the stad^ xfie sample was aboljt evenly 
divided among freshmen and seniors, ^males and females. Mean AdT 
Composite scores for four of the five institutions wef"e roughly equivalent (24, 
23. 24. 25). The mean ACT Composite s'core for the fifth institution was h^ber 
than for the .other student groups.. (29J- / • 



* This first use of the c^bjective Test suggests that it measures general educa- 
tion knowledgpfaRd skills accurately enough to use in program* evaluation. 



^ , ' • As a proxy measure for the, Battery, the Objective Test wili'be more useful 

^ ' ) for making judgments about grpupsi than for jntfivid^J students. The 

^ correlation bet\.Veen total scores of the Battery and Test Vas ,60, Correla- 
tions between subscores for thp tvyo measures, in each of the six majcrr 
^ ' argas/Tanged fcom ,37 to .48, as illustrated in Table 5. t 



TABLE 5 ^ f 

Cof relations of MeasurerrrenyiaUery and Objective Test Scores . 



^.Functioning 

totol in Social Using Using ' Gommuni- Solving Clarifying 



J Score Institutions Science the Arts eating Problems Values 

, :60 " ^42 ' 48 .37 A4 . .42 .47 



With J^urther revisits in the Objective-Test, it is anticipated that correla- 
tionsjn all six/major areas ^ill be raised above .60, ^ 
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> As ah Independent measure of general education. We Objective Test totah 
^ sTOre has an^estimateJ reliability of .65, using Rristof's estimate ot nelia- 
/ blUty for a three-part test, Thf mean scor|;iorjeniors Is Signif icantl^^ fiig^ief 
^han for fresbm^n^, There' werfe slgnlficant^differences in all the majaf area 
^'^mean subscoreeWc^ptyin^Comrnunicaling./ However. Kristof mifebility 
f^'estlT^&t^ for subs^# af? in yie 50 to .ZOirange. With further revisions of 
the Objective XWtrit Is anticipated that all subscore reliabilities will be 
^ raised to^acbeptatfle. levels/ . / ^ 

It is not likely that acD§^ahie proxy measures of.^wrjting and speaking ^ilAe 
^ deveroped. Thui insfitytions which use the Obffective Test^may wish to use 
th% Measurement Battfery Writing and SpeaKing subtestsito assess these 
areas' Ifnciusiofl&bf thpse two subtests in4he Communicating subscore la 
almost certain t® resuLt in a scale ttiat discriminates well. 

with ivfeasuremeht Battery, there is nearly zero correlation between 
:^he Objective Test total score and ^age. This suPggests that <he abilities 
bein^ rneastired are unrelated to ijmple maturation. Corretatlons with 
major area of interest and sex were^almost ?ero. 

• For the seniors who responded to bol)i the Med^remer^t Battery ind the 
Objective Test, the correlation of totaf score wiW GPA is significant/but 

y smdll (.27)- This compares with the correlation between GFA apd Measufe- 
ment Battery tqtal store ( 35). The Obj9cti\^e Test thus a^pedrs te b© 
assessing abilities not nneasured by college grades. ■ ^ \ \ 

§ The Objective Jest appears to be mpre closely ,rekat^d to measures of 
general ability than the Measurement Battery. ACT CoMposite scores fw 
students who took both tests yielded correlations of .52 for thes^Measure^ 
r^ent Battery and JO for the Objective Test. Nevertheless, half of fhe dlffer-^ 
ences in performar^ce on "the Objective Test cannot be attributed to genferai' 
ability. 
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VoeatjoniUTechnical: The purpose of this section is to present an analysis of scores obtained by 258 
Student Use^df the vocati^al-tech|iical students who tCMDk a part of the COIVIP/ACT Measure- 
' QQj^py^Qj ;ment Battery ^^hng 1977-78. _ ' J 

Measurerhent Battery |^ order to make an appfopriate comparison between the performance of 
- these vDcationahtechnical students oh the Battery and that of students 

'\ ^ \ ' attehdmg.4=y©ar colleges participating tn COMP, samples from each group 

were matched in ability, age, and sei. Ability scores (Florida 12th grade total 
scores) were available for only 81 of the vocational-technical students and 31 
of these scores w.ere below the^ility range of\the colJege sample. There- 
fore, a random'^sample of 50 college seniors was selected, and matched with 
the 50, vocations-technical' students in ability, age, and sex. A ^able of 
concordanae was'^used to equ^ate.the Florid^ 12th grade total ability scores 
) with ACT composite'scorel for coirege seniors. Th^ two samples thus 
n^atched were of the same size (N ^50 each). The average age of aach sample 
, \y - v^7as identical (23 yfears), and tte^ge range was also roughly equivalent (19- 

37 years for the vocational-technical group and 20-34 years for the college 
' ' . seniors). M the time the two grp^s were seniors Jn high scho'ol, they had 
reached roughly equivaleryt levefs df general academic development, based 
oVi an average ACT Composite icore of 20.36 for the college group and a 
Florida 12th Grade Test score average for the vocational-technical students 
^-^qui^alent to an ACT Composite score of 18.94. The ACT Composite score 
' range for the college san^ple was 12-26 and the range for the vocational- 
" technical sample was 10=2i. * 



The two samples had roughly the same number of males and females, with 
- ^the college sample being 44% male and the vocatfonahtechnigaL sample 46% 
roale. The college sample was drawn from six different colled^s participrating 
n COMP and the vocatioHoUtech^cal sample represented eight of the 
; Florida vbcational-technical centers. Given these efJorts to n^atch ability, sex, 
and age, the interpretations which follow are based ^on thepssumpiton that 
the two samples are comparable. 



All 15 sets of stimulus materials and questions in th% Battery were 
administered to'college students at colleges participating in COMP field tests, 
whereas onl^ 12 of these sets were given to the vocational-tefchnical students. 
To make appropriate comparisons, scores were recomputed using the same 
12 sets of responses for the 50 college seniors in the matched comparison 
group. The average scores obtained by the two samples are displayed in 
T^ble 6. 

f 



TABLE 6 

Mean Scores for the^OMP/ACT Measurement Battery (Modified Form) 



Functioning , „i n 

.Total In social Using Using Communl- Solving C ar tying 

^ Sceri Institutions Seltnct the Arts eating; Problems Values 

-College ' ^ . j . ^ 

44.52 7 70 ^ 5.2«' ' _ 6.94 8.10 10,14 6.36 

Matched 

^ 'Group of . . ^ " 

f Voc/Tech ■ " I ■ 

'27^2 4.52 -.2.00'- 3,6a 6.56 7.02. 3.50. 

- ^ ' " ' , , ". ' - 

Total • . « 

GrouprOf 

Voc/Tech ' 
Students 
lNi258) 23 81 



3.86 1.76 3.19 6.00 



6.3d 2.70 



The college seniors obtained-f^an scores substantially above those of the 
vocational-technical students. Furthermore, the vocational-technical stu- 
dents responded, to a modified version of the Meafyrement Battery, omit- 
ting activities 5 10 and 15, which constitute over one-third of the questions 
judged to be the most difficult and complex in the Battery. Had the voca- 
tional-technical students responded to the total Measurement Battery, the 
differences in mean scores might have- been even greater. Since the two 
groups were judged to have the same level'of general educational develop- 
ment at the end of high school, we tentatively conclude that the college 
experience is responsible for further development of the college seniors, Yet 
because a number of factors influence educational development, these 
results rci'erely suggest developmental differences resulting froi^ different 
educational programs. Further research on thts: issue Is planned for the 



future. / 



One further interesting bit of Information appears in the'data of this study. 
Assuming that a total score at the 25th percentile of the college group is com- 
mensurate with the general education competency expected for the award- 
ing of a B A, degree; 21% of the 258 vocational-technical students could be 
judged as already meeting the intent of many undergraduate general educa- 
tion programs. Table 7 develops this data further. 



TABLE 1 

Percentage of Vdcational/TecHnlcal Students Scoring 
at or above the 25th Percentile for the Comparison 
/tfjoup of pollege Seniors 



Raw 

Score at 
25m %tiie 


Total 
Score 

32 ^ 


Functioning 

in Social 
Institutions 

6 


□sing 
Science 

) 

-^^ 3 


Using 
the Arts 

4 


Communis 
eating 


Solving 
Problenii 

7 


Clarifying 
Valuis 

4 


Matched 
C3roup of 
Voc/Tech 
Students 
(N^50) 


32% 


34% 


28% 


O 

44% 


72% 


60% 


30%/. = 


Total 
Group of 
Voc/Tech 
StudBntS 
(N -258) 


2 1 % 


24% ^ 


26% 


38% 


59% 


45% 


26%^ 



Since the vocational-technical students did not take the entire Battery and 
these 258 students may not be representative of allf vQcationaMechhical 
students. It is difficult to estimate how many such students may already 
possess general education skrlls and knowledge, as measured by the Battery, 
at a collegiate leveL However, the evidence provided by this study suggests 
that many such students may exist. Should these students wish to transfer to 
a program leading to a liberal arts. degree, it would appear appropriate for 
'them to receive credit for these skills. 



^4 
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Mature Adult 
Performance on 

' COMP/ACT 
^ Instruments 




Since a major goal in developing the three„COMP instruments has been Jo 
maas^re the ability of persons.to function effectively in adult society, it seems 
appfopriate .to examine the performance of. adutts on th^ Measurem^en 
Battery Objective Test, and Activity Inventory. Such Investigation of adult 
-performance should provide an .indication, of^the predictive validity of the 
instruments for effective functfoning in adult roles. Of three such studies 
conducted in 1977-78 two involved 58 adults in Mtlwaukee and 30 adults m 
Miarr^i The third Involved 41 college students at least 25 years old enrolled m 
■five of the instilutlons participating in COMP. These studies vvere of an 
exploratory nature and were designed to gain a very general idea of the effec- 
tiverjess of the three COMP Instruments, 

The adults selected to participate were recommended by leaders of com- 
munity vQictfrteer organizations as those effectively functioning at an amateur 
level in one of the ■ following roles; clarifying ' social values, solvjng 
technological problems, or communicating about the arts. The adults were 
asked to select one of the following areas as being of greatest interest to 
them- social science, natural scienci, tfrwts/hum|nitles, In 80% of.the cases, 
the area of greatest Interest selected by the partidpant matched the role f^r 
whichVhat Individual was Identified. Thus, the rolesVhich most interested the 
participants were also, by implication, those for which they were beat 
qualified The older college students were also asked to identify which area 
most interested them so that the following comparison could p made. 
0 

All participants completed the open-response version of the Activity 
Inventory which was then scored using rating scales developed m he 
nroiect These adults definitely tended to score highest on the section of the 
inventory that corresponded to their self-reported area of greatest mteres 
(-^statistically significant above the .001 level with a contingency coefficient of 
67 out of a possibre .a2)\ Furthermore, the mean score for those identi ied 
with social science on th^Functioning within Social.lnstitutions (FSI) sectio^ , 
of the inventory was higher than the mean scores for those identified with 
natural science and arts/humanities, Those identified with natural science 
had a higher-mean scsre in Using Science and Technology than did those m 
social sciences and arts/humarilties. Those Identified with arts/humamties 
had a higher mean score- In Using the Arts than did those \n the social and 
natural sciences. Findings from these initial studies offer evidence of he 
validity of the Activity Inventory in identifying effectively functioning adults. 

A- a result of this evidence, three or four greatly expanded studies of the 
validity of the Activity Inventory are planned for 1978-79. Also, the responses 
provided by the two adult studies, together with those provided by 470 
students have aided in developing an objective version of the Inventory as 
well as the rating scales used to score the open-response version. 



I ne next step was to determine whether scores on the Measurement Battery 
obtained by adolescent^students might be predictive of futureJunctiorrmg in 
adult roles, by examining the cor^relation between Battery and Aatlvlty 
Inventory scores. The Milwaukee and Mi^mj adults responded to only" one- 
third of the Battery. Everr so. the totai scoreg^or thPs partial Battery showed 
rather high correlation fr-.43) with Inventory tot^l scores'for these adults. 
Ag^m. this evidence should encourage further studies of adult performance 
on the Battery during 1 978-79. In the study of 41 students over the age of 25 at 
five COMP participating college, a correlation of .69= was obtained when 
these adults mqk the entire Battery as well as the entire Inventory^ 

The third sfep in this study was to determine the degree of correlation 
between scores on the Objective Test and th^e Battery. Not only would this 
yield mformatlon about the degree to which the Test could be used to predict 
SQores on the Battery, .but it would indicate how well the Objective Test might 
serve as an Inexpensive proxy measure In evaluating the general or liberal 
education program of a college. Results for the three groups varied widely. 
For the Milwaukee adults, a correlation of .46 was obtained between Test total 
scores and Battery total scores. For the Miami adults, the correlation was .59 
Here again. It is important to remember that these adults took only one^third 
^of the Battery In the case of the 41 older students, the 17 who took both the 
entire Battery and Objective -Test showed a correlation between t^^test 
scores of .74. Although variable, these correlations suggest it is possible to 
generate viable proxy measure's. ACT staff are greatly^encouraged and will 
continue development of the Objective Test in 1978=7^ 
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otRating Speaking ^''^tSt ^Sgf level do not gehtrally result in- high levels of agreemimt 
; ind Writing Skills ^^t^nr^l^. Furthermore, the existing measures are o^ten more hke 

SSL Indents do in classrooms than tasks a 
. . nves' ^n^^elr validity as accurately reflecting abilities .mROrtant to 

effective functioning is also open to question. 

' Dart of the College Outcome Measures Project, direct measures of Speak- 

• • ■ ; ^ a^w'ltSe been developed Jhat us^ 

^ • . ^ V ^lledcolleg^Hlt level skills. These are included as part of the COMP/ACT 

' ' _ " ; - Mfeasurement Battery, ^ . ■ 

in two test-retest studies of this Battery^ total t^t score 
.89. and subscore , reliibilities of .66 to ,88. ^jere ob^ned, R^ M^y 
coefficients for the Communicating subscore v^ere .80 and .79. When two 
■ ^ . ■ lompoSl of this subscore. Speaking and Writmg, exam.ne^^ 

. . : ' . however reliability coefficients were found to range from .37 to .72, The 

h summarized in this section were Initiated to determine whether 

: . V' - . . . ^emenl^ln iting procedures plus training of evaluators would ra^ 

interratar agreement to acceptable levels, , , 

' : / . using data from the test^retest study of the CQMP/AJT M^umm^^ 

W mentioned above, two tl-ained raters independently ^ted 74 se^s of^^^^^^^ 

■' t • - . resDonses and 65 sets of written responses.. In the Speaking subtest 

comoosed of role^playing tasks in each of the three content areas of social 
' • sc^^liaLS^SencJ, and the arts, students were given 24Jiou^ 

; ■ • audSo^vSeotaping three 3-minute responses. The writing subt^n 

- ?h,^e role p^^^^^^ 

;^^riSo Ses Of writing to compose three letters directed to^various 

audiences. ' 

Raters were required to apply qualitative rating scales, defining five levels ^ 

, pm^de^cy^ three broaS criteria, to each of the three oral or wn«en 

. ■ S^Guideli^s were provided to minimize various sources o^ bias. n 

' So As a tralninrprocedure. the two raters Involved m theSpeaking and 

wS^u^^s we^ given practice in rating ten student ^a^f t° o 
^ ' ■ ^e h?^e writing or speaking tasks. They were then asked to discuss the^ 

^^r^swi^Soth^ and Mtify any discrepant ratings. .Finally, the^ 
^' ' SeSb^k on how othe4 had rated the responses. Differences in evela^ 

• . ■ ^ - - ■ weH as spr^S of rating were pointed out and strategies to compensate were 

.V d^c^ed ™°practiL rating session requ 

the two studies, " ^ 

Two approaches to studying interrater agreenrient were used. First 
. , ^reVolnt rating represented continuous data, a Pearson produ^ct-m^^^ 

correlSn v^as calc^ 

aN th ee student responses.. Second, the percentage of total agreement and 
..agrSlvSthln one category were calculated for ratings 
• • ' ■ responses combined; Percentages of agreement on a pass/fail basis at each 

level of proficiency were also calculated. 
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For the Speaking subtest, the correlation between two independent ratings by 
trained raters was,83. Percentage of exict agreement about which of the five 
levels of proflcierr^^a^^ student response shoufd receive was 51% for 
trained raters, or doQb^he figure for untrained raters.. Trained raters 
achieved 94% agreeniem"Sn>«tings wlthl^ one category on the rating scale. 
This resulted In extremely acc Jrate ^tlngs of student responses at minimally 
acceptable and advanced levelrB^oflclency. When evaluating speakers on 
a pass/fall basis, trained raters agreed 99% of the time about whether perfor- 
mance fell above or below the lowest standard. At the second level of 
proficiency, 87% agreement was achieved; at the third level, 77%; and at the 
highest standard, 83%, Thus, it appears that Speaking can be objectively and. 
reliably rated by trained raters using COMP/ACT materials and rating pro- 
cedures. The validity ofsuch measures In relation to effective functioning in 
adult roles is currently being studied. 

For the wHting subtest, the correlation between two independent tatings by 
trained raters was .76, Percentage of exaot agreement about which of the.flve 
levels of proficiency each student response should'recelve was 35%, Trained 
raters achieved 84% agreement within one category on the rating scale, and 
the accuracy of rating student responses at minimally acceptable' and 
advanced levels of proficiency approached that of the Speaking study. When 
evaluating Writlna on a pass/fail basis, trained raters agreed 91% of the time 
about whether p#formince fell above or below the lov^est standard. At the 
second ^level of proficiency, 72% agreement was achieved; at the third level. 
71%; and at the highest standard, 82%v This pattern suggests thai- the lower 
correlation obtained In the Writing study is due to disagreements at 
intermediate levels of proficiency. With further refinements In the rating scale 
and training materials, It is likely that even higher levels of Interrater agree- 
ment can be obtalhed. - 

These studies demonstrate that Speaking and Writing can be measured 
reliably. Using such measures, normative data can be generated for cdllege 
freshmen and seniors. These data will be highly relivant to' the current 
emphasis »on basic skills at the public school level. 

A more imme^te and practical use for these measures Is In examlnmg and 
improving college programs which aim tO;deyelpp Speaking *d Writing 
skills; Many postsecondary institutions are currently reexamining their 
general education programs and the meaning of a liberal arts degree, and 
these dat^ might aid in |tudy!n^ alternative Instructional strategies. Using 
such measures, students could also be screened and more appropriately 
placed In programs addr|ssing their needs. For Institutions wishing to verify 
the development of Communicatlor^skills^at acceptable levels, measures 
Such as these couJd be used to assess growth and certify competence. 
Finally, the assessment and rating procedures offer a model for college 
, faculty to use in developing apd reliably rating other open-ended measures of 
skills and knowledge. 
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The COMP Adviaory and Evaluation Panel is compoaed of Dn George L Hall, 
Exmoutlve Director, State Board of Directors of Community Colleges of 
ArizonB; Dr. Wayne H, Holtzman, President, the Hogg Foundation for /Cental 
Health; Dr. fi/lorris T, Keeton, Executive Director, Council for the Advance- 
ment of Experiential Learning; and Dr, P$trlcia A. Thrash, Associate Directon 
• North Central Association of, Cblleges and Schools Commission on Institu- 
tions of Higher Education. 

The Advisory Panel of the COMP/ACT project, fflnded by the Fund for' the 
Improvement of Postiecondary Education, has asked me to report to you on 
the paners evaluation of the project to date. In doing so, I have,the benefit of 
written comments from the other members of the panel as well as the 
experience of our meetings and the documents produced durlhg.the project. 

The purpose of this project was tcf generate a model solution to a most in^por= 
tant problem In higher educationrthe lack of a valid, reliable, and cost effec- 
tive process (witri implementing tools) for assessihg results on a wslH 
conceived and defined set of general education outcomes of college educa- 
. tion (baccalaureate levelj. The desired outcomes were seen, not simply as 
knowledge acquisition nor simply as general academic aptitude, but as the 
capabilities which effectively functioning adults would need for a personally 
rewarding and socially useful life In this society. It was a high risk endeavor to 
seek to generate such a process, and the risk was compounded by the 
difficultifs of assuring acceptance and use of the model by Institutions of 
higher education. 

The purpose, high risk though it was, has nevertheless -been achieved in the 
Judgment of the panel. The dissemination of uses of the model beyond the 
experimental group of institutions still lies ahead; but the most jdlfficult 
hurdles— develpping a suitably definition of general education outcomes; 
enlisting an appropriate array of experimental institutions, achieving relia- 
bility In the measures, achieving valid measures, and doing these things in a 
way that'would promise cost effectiveness— have been overcome. The basis 
for this judgment is here presented in the form of answers to four questions: 

1 ) Do the outcomes selected for assessment meet the tests appropriate to the 
purpose of the project? a) Are they learnable through baccalaureate pro- 
grams of general or liberal education? and b) Do they give promise of 
appropriate correlation with effective functioning in adult society? 

2) Is It Ukely that the three assessment instruments developed in the project 
will prove to be valid, reliable, and cost effective? 

3) ' How effective have the dissemination efforts been? a) 'Are ihe com- 
munications clear, relevant to both the purpos© of the projept and the likely 
concerns of user institutions, and thorough in dealing with ^the available 
information? b) Have thp dissemination efforts been appropriate to the 
purpose of eliciting eventual wider use of the project outcomes? 

4) Do the probable benefits of the project render the outlay of funds cost 
effective? 
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1, Appropriateness of the Selection of Outcomes to be Measursa. 
A number of Blements enter Into the definition of what is appropriate as a 
selection /of outcomes to be measured in this project: a) For traditional 
definitions of what the outcomes of general education should be, there is no 
need for new measures. Reliable ones now exist. While the selection of 
particular test items might be questioned, there are already arrangernents m 
being for periodic review/The relevance of these traditional definitions to. 
wh^i is needed by the society from college graduates is also under inoreas- 
inn question. The selectioti of outcomes to be njeasured in this project there- 
fore needed to reflect a different concept of the intent of general education^ 
b) The most promising line of inquiry or exploration appears to be one which 
speaks to the capabilities achieved by students rather than the particular 
knowledge content mastered. But what capabilities? Traditional measures 
have correlated best with success In later college studies, much less wel wi^h 
effectiveness In. adult roles in work. Little has been learned about he 
Dredictability of effective functioning of college graduates in other Jater life 
roles The decision to try to define the capabilities of "ap effectively function- 
ina adult" and to develop outcome measures with significant predictive value 
for such functioning -v^as therefore one likely to be useful to both the institu- 
tions of higher education and their graduates,, c) The ideal choice of 
outcomes would be one which made possible measurertients which could 
both facilitate program evaluation and improve the usefulness of assessment 
information to both the students (graduates) and their employers and peers_ 
d) The outcomes to be measured should differ significantly from related and 
already measurable changes that occur during the course of college years;- 
e q from simpJe maturation.' from academio^aWitudes ^s defined in existing 
measures, and from achievement or knowledgb t|st8 such as thoSe in the 
Graduate Record Examination series. 



The six outcome "areas" chosen for measurement con^t of three capa 
areas (communications capabilities, problem solving capabilities, and value 
clarification capabilities) interacting with three arenas for application (tunc- 
tioninq within social institutions, using science and technology, and using the 
arts) to yield nine areas of ability to function in adult roles.^These outcome 
areas seem to the panel an appropriate choice using the criteria just stated, 
and the findings of the field tests of the COMP/ACT Measurement Battery 
support^he judgment that these outcomes can be. and are, affected by 
coneae education which Is designed to have those effects and do correlate 
significantly with the characteristics of effectively functioning adults. 

. Sprlnq and fall field tests were conducted in 1977 on Forms 1 and 2 respec- 
tiveiy/of the Battery. "The results of the first field test show a clear P/oprei- 
sion of growth across four years of Gollege. . , The results of the fall field 
test for a new form of the Battery show highly significant differences 
between freshman and senior means. ... In both field tests there is a 
nearly zero correlation between age and total test score. No significant 
differences in total test score were noted by sex or major field. 

'^^tTmaZom Is exoBrpted from an earlier version of the first stpdy desorlbed In^hja report 
4lM977 fielS Trtals of the COMP/ACT Measurement Battery, Form ll," ongmally circulated to 

participants in the project. 
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• A late/ 1977-78 Biudy compared the performance of graduates of post- 
ieoondary vocatlonahtiohnlgal programs with that of baccalaureate 
graduates of the project colteges On the C0MP/ACT Sattery,. (See the 
fourth study deicrib€[d In this report, 'VocationahTechnical Studeg^ype of 
the COMP/ACT Measurement Battery," for details:) The baccJI^rtlite 
graduates scored substantially higher than did the vocatlonaMechnlcal 
ones. Panel members, however, insofar as their com'ments have been 
received on this study, So not regard it as well designed to provide the kind 
of matching groups of students required for a deciiive evaluation nor was 
the control of other factors strong. There is need for4hi time and other 
conditions that Will permit pre- and post-training measures.and other steps 
to reduce the possibilities that artifacts will enter into the results. 

It will always be possible to debate as to what outcomes colleges should be 
trying to generate In their general education programs, what priorities the 
different -objectives should have, which of those such as here addressed can 
be "taught^deliberately,^! i^^^ questions. In this instance a sizeable group 
of colleges, which are known to represent interests shared by a larger 
company, had the opportunity to try to clarify what they most valued among 
the possible outcomes to be sought and assessed. Given the currept 
challenge as to whether college education is really worthwhllelnd the exteh- 
sive pressures to answer that quMtion primarilyjn terms of Job- and career- 
related concerns, the, choice madte by these colleges and applied in this 
project is a significant effort to ^larify and revitalize the commitment to' 
complementary values. It reaffirms the idea that college education should be 
judged by, the quality of life to which it contributes and that this quality Is a 
matter of the development of Interests and concerns and of behavior apply- 
ing those concerns in- participation in the adult community. 



2. The Validity, Reliability, and Cost EffQdtivQness of the Assessment Instru- 
ments and Processes, ' ' 

The most appropriate general answer to^ the questions on the technical 
adequacy of the COMP/ACT Battery Is that It has achieved reasonable 
validity, reliability, and cost effectiveness. 

The validity of such instruments ^can b| at beat only relative. To obtain a 
strong confirmation that the results gf^e of this Battery will measure what 
really makes for effective fMnctioning.^adults is a formidable undertaking. It 
cannot possibly be completed until en^gh time has passed to have the early 
examinees grow into such adults andajndergo further testing. The project 
has, since time must pass before sufh testing can be done, chosen inter- 
mediate clues and criteria which the panel views is reasonable ones. The face 
validity of the measures is good, VVe have not been able to suggest better 
ones> Of rtain findings to date are reassuring. For example, the Ba^ttery does ' 
not measure some things it was meant not to measure. It Is not simply 
measuring college grades. The correlation between age and total test scores 
is near zero (the Battery Is not sf^pl^measuring maturation)^ The Battery 
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measures something other than general aptitude and a^'n^^tf^^^^J^P 
leaves over half of the variance In performance unexplained on the total ■ 
Ses) On the positive side, the studies done with older adults identified as 
eSe in the Las under study showed a statistically significant correla- 
'oSeerf high scoring on the corresponding tests and the .dent^^ar^as . 
of their greatest effectiveness. Cultural bias does not appear to be working in 
the results. . . i 

With regard to the reliability of the tests, three kinds of reliabimy appear to 
' have been achieved at suitable levels. Interjudge agreement in the ratings is 
very high. On Internal consistency and homogeneity of scores, the evidence 
^ r^t as extensive, but what there Is looks good. On test-retest ^tu^^^e 
iliablJities are quite high. For such an early stage of test development, these 
^eluite heart^ing r4lts, (See the second study described m this report 
"A Test^Retest Study of the=GOMP/ACT Measurement Battery, Form II, tor 
details.) . ' ' ; . . ^ 

in evaluating the reliability of the instrurrients and t^e assessment process 
caiful attemion must be given to the purposes ^vhich the resul^ a^^t^ 
used It must also be remembered that the Battery is still m the P opess c.f 
. being refined and that further development of both.the instrurTient arid the 
trainmg of Judges may result in improved reliabilities. In the fall 1977 study of 
he Balery seven purposes are identified for which the results might be used 
Of that of program evaluation and planning the Battery already provides quite 
adequate levels of reliability. The remaining functions have to do ^^P^^^ 
ing individual performance and admitting people to opportunities or placing 
hlrnwhhin programs or enterprises. On these functions a distmc^ns^u^d 
be made between functions having to do with selection annong cor^petmg 
candidates and those having to do with guidance or counseling for the indi- 
Sll^e reliabilities of the scores obtained with the Battery are alre^ 
Approaching levels adequate for the selection functions. P^^J^^ • 
of ^e fact that It Is difficult to find alternative means that approach such leve s . 
of reliability. On the functions related to guidance, there is a considerable 
way yet to go before the Battery scores can be regarded as adequate. 

The studies and reports made on the Battery to date give a clear picture pf 
progression from the first roughihewn construction efforts to the present 
n.or« refined condition of the instruments. The trend gives a strong sense that 
the task of reaching adequate sophistication In the instruments can bf do^ 
The researchers have made impressive efforts to expose the possible sources 
of error in rating. They have been candid in reporting needed 'mprovements 
, as well as encouraging flndlngs.'The, likelihood of locating problems and 
removing them therefore seems strong. 

The panel has been concerned that measures developed in this project 
should begin to get at the higher levels of, 'nte'lec^^^' p^^^^^^f,^^, 
problem-solving skills as defined, for example, in the Bloom and Broder 
Komy of c^nitive objectives. This task is a difficult ^^^^^^ 
eel that the pFoject staff has made some headway on it The Activity 
nventory is one expression of this effort, may prove to be only a begmrimg 
b^may also prove useful as a "stand-alone" Instrument, This whole effort 
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* ' : may make a contribution to the meaiurement of Jht contributions, of 

? ^ experiential learning to collegt .education. We also see. the effort, after 

• ' : achieving a reliable measure of a different kind of colleger 

' ' ^ ^ traditionally measured by objective tests, to find an Objective test which will 

' serve as a proxy measure for this neNv type of outcome as a worthwhile one'. 

As to the cost effectivenesi of the instruments and processes, the panel had - 
^ad grave misgivings early In the proJect^Wa'are Impressed with the progress 
made on this concern. The cost estimate of some^25 per student In the most^ 
^ recent study appears to be accurately developed. When one considers the 

$25Q0-$7Q00 per year spent in the general education of studentj, arid notes 
that up to the fulMime equivalent of two years may be devoted to the ge 
education requirements .of a college, an expenditure of $25 per student to 
assess the outcomes appears very small Indeed, Further' refinement of the 
instruments and procedures Is warranted; and If It shjould prove twice or more 
' times as costly to obtain results reliable, for Individuil guidance, the cost ' 

' / ' would be well Justified, The costs of such assessment In portfolio develop- 
ment and evaluation, though yleldlng^more Individuaiized and defailed 
specification of learning outcomes, Is so much greater (comparable to the 
? cost of an entire course or two) that the COMP/ACT Battery, alone or In 

corhbination with such other means, would be a significant contribution to 
quality assurance and to effective educational planning and guidance. 

In connection with both the questions of reliability and of cost effectiveness, 
it should be noted that failure of some of the participating Institutions to enlist 
sufficient nurnbers of subjects has yielded numbers insufficient to permit 
useful interinstitutional comparisons on achievement levels on the learning 
. outcomes measured by thfe,COMP/ACT Battery, Ultlmatajy It would be highly 

, useful to develop information suitable for such studies. 



3. EffectivBness of Dissemination Efforts. 

Insofar as the question of effectiveness of dissemination efforts refers to the . 
objective of spreading the use of the COMP/ACT Battery and affecting the 
design of curricula to reflect objectives of the kinds measured by the Battery, 
it is premi^ure to try to evaluate this effectrveness. The effort to enlist this 
^ider participation would pot have been appropriate until the recently 
completed studies of validity and reliability were completed. That work Is. 
accordingly just now beginning. 

It Is nevertheless possible to comment on communications and dissemina- 
tion efforts which have laid the groundwork for effective dissemination of the 
kind envisaged. The panel has seen communications addfessed to participat- 
ing institutions, to the panel itself^ and to others seeking to understand what 
is going on even before the opportunity for participation arises. In general, 
these communications have shown clarity, thoroughness, relevance to their 
functionsPand good applicability to the needs of the addressees. Both partici- 
pant groups and the Advisory Panel f\ave been, at the outset, provided clear ^ 
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statements of their respective functions and of the purpo^ei of particular . 
meSs and stages of work. In addition, there has been tiSSrough reporting 
afSSSs wVboth What happened in the meetings and what folloyv up 
aSitl^t^ic^red or were expected; The meetings themselves have 
been efficiently conducted. 

Dr Patricia Thrash states well the experience of the /^Mvisory Panel wi^^ 
s^afi^^ in these terms: "The meetings were focused; ^«ter.i^ was^le^ 
and applicable. The 'hands on' experience of the December 2-3 
meetiffg^as well as the opportunity to meet participants v^i an excellent 
h^Stion. The Marctjl=2. 1977 meeting between the Advisory Panel and 
he ACT staff provided an eKcellentiaPortunity to review the pr^Tess^^^^^^ 
project and to hear pro|ectlons for the future. The September 9, If T meeting 
oHhe Advisory Panel and project staff brought panel members up to date on 
p ani fS fieS testing a new measu«inent battery. The written mater^ 
since that time (and especially in May and Augus ) ^^^^'^^ 
and thorough. Even withoyt tacHo-face meeting. I feel fully apprised of th& 
project's progress."' - _ 

^The project has been widely publicized, explained, and discussed among 
consti uSicles likely to contribute to the success of '^^^-^^^^ 
efforts Such occasions as the annual meeting of the North Central Asiocia 
tion (1978). the assemblies of the Council for the Advancement of 
Experiential Learning, (twice in 1977), and a meeting of the Council ort Pc^ 
seloncLry Accreditation (1978) have been used to spread the word of^the 
p^bject. 7he project was treated by Aubrey Forrest in "Competency-Based 
Assessment in Postsecondary Educatlon^Some Issues and Answers.^ in the 
nS^ SS^y (Fall, 1977, Vol. 52. No. 2. pp. 322=326). T^e pmiect, m 
summa,^ is widely known. Its use, of participating institutions from the 
throughout development has contributed both to the ^^y^ 
communLtlons and to the acceptance and the likely receptivity 

battery as opportunity is opened to additional institutions. 



4. Cost Effectiveness of the Project.' 

Paralrasmqpv' peorge L. Hall of the Panel, COMP was a risky project. 
ThS^^lSellnty'nor any guarantee that valid ^-^J^^^^;^ 
tools would result.1t seemed almost too mufch to expect eMfen that the several, 
vastly different participating institution^could collecth/ely S^ad^ 
definitions and measures of outcomes. Hence, members of the panel had 
som^^epticism about the project In early meetings with the 8^ ^^^^ - 
tlonal representatives. This skepticism has gradually changed to belief. It is 
^e paS^thinking no«v that t^e majo. difficulties of the projfct have been 
o^r^ that tiell tests are very satls?kstory, and that the ne>^ 
important in the dissemination and further development of the project. 
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DiT^hrailr~expresi8s^thls^same~Judgmen 

invorved Jn postsecondary education need accurate inforrriation about 
general and liberal education outcomes in order to make sound educational 
decisions. This skill in assessing outcomes is still In its infancy; there is much 
more talk than evidence that it can be done. In this project, for the first timej a 
serious and workable attempt has beer) made to do what all of us— educators, 
students, accreditors, financial supporters— say needs to be done. The instru- 
ments thus far designed ^are cost-effective Both in time and money terms. 
They are cpmpjementary.jthe promised continued, commitment and suppoft 
of ACT should serve as a guaranteajo FIPSE that these Instruments will fini: 
their^way into the mainstream ©f postsecondary educatipn. They will be of 
great assistance to IndivldualSi at all stages of their development from college 
onward^ They will be of extraordinary value to postsecondary institutions. I 
was impressed by the specific primary uses cited in the Fall 197*7 Field Test 
Report, Each of the seven speaks to a concrete' and urgent area of need ^ 
^general or liberal education program evaluation and planning; academic 
advising; academic placementf credit to , adults for experiential learning; 
certifying students for graduation; screening students for employment; and 
screening students for graduate or professlonaf training). . . .The project 
involves a timely effort in a significant. . . area of education. I believe, , . that 
'literally hundreds of postsecondary institutions would enthusiastically take 
advantage of the assessment instruments and procedures that result from this 
project.' I know of at^east 200 North Central Institutions that would be 
receptive to something as concrete , ana enabling as these Instruments are. 

"At its best, the COMP/ACT project represents^a significant breakthrough at 
the best possible moment. If subsequent development and broader exposure 
of the instruments yield less sanguine results, a'signif leant step forward will 
still have been made. I am not a researcher; I am an edufttional practitioner. 
My own experience tells me that students and Institutions need these instru- 
ments badly. The evidence of their effectiveness is. compelling. 

The panel wishes also to record its appreciation for a dedicated, hard- 
working, persistent, and cbmpetent staff and core of Institutional representa- 
tives. The Fund for the Improvement of Postsecondary Education is fortunate 
to have entrusted this difficult venture to such good hands. 



^Letter from Patricia Thrash to Morris Keeton, September 4, 1978. 



Alverno College 

Brlgham Young University ' * ^ 

Colgate; University 

College of Du Page ^ 
Delaware County CommMnity, College 
Florida A&M University 

Florida International University * ^ 

LIndiey-HoRkins Technical Edueatlon Center 
Lively VocaflbnahTechnlcal Center 
Mars Hill College . . 

Memphis State University ^ 
Miami-Dade Community College 
Michigan StatJ University 
Mid-Florida Technical Institute 
Okaloosa-Walto^ Junior College 
Our Lady of the Lake University , 
St, Louis University ^ 
St. Petersburg Junior College * . 

; Sarasota Vocational-Technical Center ^ 
Seminole Community CoHege 
South Brunswick High School 
State University System of Florida ^ 
Tennessee Higher Education Commission 
Tennessee Technological University 
University of Nebraska-Lincoln ' ^ 

University of Windsor - . 

Washington-Holmes Vocational-Technical Center 
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